Performance improvement of a capacity optimized storage system using a performance segment storage system and a segment storage system

ABSTRACT

A system for storing data comprises a performance storage unit for storing a data stream or a data block in. The data stream or the data block comprises one or more data items. The system further comprises a segment storage system for automatically storing a stored data item of the one or more data items as a set of segments. The system further comprises a performance segment storage unit for storing the set of segments in the event that the stored data item has been stored using the segment storage system.

This application claims priority to U.S. Provisional Patent ApplicationNo. 61/277,271 entitled PERFORMANCE IMPROVEMENT OF A CAPACITY OPTIMIZEDSTORAGE SYSTEM filed Sep. 22, 2009 which is incorporated herein byreference for all purposes.

BACKGROUND OF THE INVENTION

Segment deduplication storage systems are efficient in terms of storagecapacity. In a segment deduplication storage system, a data stream ordata block is broken into segments. Segments are stored in the eventthat a segment has not been previously stored. In the event that asegment has been previously stored, a reference is stored to the priorstored segment. Information is also stored on how stored segments areused to reconstruct the data stream or data block. However, segmentdeduplication storage systems are in some cases too slow for storing andretrieving data stored in the system. Storing can be too slow becauseprior stored segments must be searched to see if an incoming segment isthe same as a prior stored segment. Retrieving can be too slow becausedata must be reconstructed from a number of data segments storedpreviously.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1A is a block diagram illustrating an embodiment of a storagesystem.

FIG. 1B is a block diagram illustrating an embodiment of a storagesystem.

FIG. 2 is a block diagram illustrating an embodiment of a segmentstorage system.

FIG. 3 is a block diagram illustrating an embodiment of a performancestorage system.

FIG. 4 is a flow diagram illustrating an embodiment of a process for astorage system.

FIG. 5 is a flow diagram illustrating an embodiment of a process for astorage system.

FIG. 6 is a flow diagram illustrating an embodiment of a process forproviding requested data from a storage system.

FIG. 7 is a flow diagram illustrating an embodiment of a process fortaking a snapshot of data item(s) in a performance storage unit.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess; an apparatus; a system; a composition of matter; a computerprogram product embodied on a computer readable storage medium; and/or aprocessor, such as a processor configured to execute instructions storedon and/or provided by a memory coupled to the processor. In thisspecification, these implementations, or any other form that theinvention may take, may be referred to as techniques. In general, theorder of the steps of disclosed processes may be altered within thescope of the invention. Unless stated otherwise, a component such as aprocessor or a memory described as being configured to perform a taskmay be implemented as a general component that is temporarily configuredto perform the task at a given time or a specific component that ismanufactured to perform the task. As used herein, the term ‘processor’refers to one or more devices, circuits, and/or processing coresconfigured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

A system for storing data is disclosed. The system comprises aperformance storage unit for storing a data stream or a data block in.The data stream or the data block comprises one or more data items. Thesystem further comprises a segment storage system for automaticallystoring a stored data item of the one or more data items as a set ofsegments. The system further comprises a performance segment storageunit for storing the set of segments in the event that the stored dataitem has been stored using the segment storage system

In some embodiments, a system for storing data is disclosed. The systemcomprises a performance storage system for storing one or more dataitems. A data item of the one or more data items comprises a data fileor a data block. The system further comprises a segment storage systemfor storing a snapshot of a stored data item of the one or more dataitems in the performance storage system. The taking of the snapshot ofthe stored data item enables recall of the stored data item as stored atthe time of the snapshot. At least one newly written segment is storedas a reference to a previously stored segment.

In some embodiments, a system for storing data is disclosed. The systemcomprises a performance storage unit and a performance segment storageunit. The system further comprises a determiner. The determinerdetermines whether a requested data is stored in the performance storageunit. The determiner determines whether the requested data is stored inthe performance segment storage unit in the event that the requesteddata is not stored in the performance storage unit.

In some embodiments, a storage system provides performance by storingdata in a performance storage system. The data stored in the performancestorage system is automatically stored in a storage capacity optimizedstorage system (e.g., a segment storage system, a deduplicated segmentstorage system, a compressed storage system, etc.). In variousembodiments, automatically storing occurs in the event that theperformance storage system is full, that a file is finished being used,that a time period is completed, that a scheduled transfer is indicated,or any other appropriate criteria for automatically storing. In someembodiments, after automatically storing a data item in the capacityoptimized storage system, the data item may be removed from theperformance optimized storage system. In some embodiments, in the eventthat the data item is desired to be available with high performance to auser or another network attached system from the storage system, thedata item is transferred back to the performance storage system. In someembodiments, the data item transferred back to the performance storagesystem is stored in a capacity optimized manner (e.g., as deduplicatedsegments, compressed, etc.).

In some embodiments, in the event that one or more data items is/aredetermined not to be needed for high performance access by a user orother network attached system from the storage system, the one or moredata items is/are removed from the performance storage system. Prior toremoval from the performance storage system, it is determined whetherthe one or more data items are stored in the capacity optimized storagesystem, and in the event that they are not, the one or more data itemsare stored to the capacity optimized storage system.

In some embodiments, the advantages of segment deduplication storage,which is efficient in terms of storage capacity by storing newlyreceived segments as only references to a previously stored segment inthe event that a received segment is already stored, is improved usingperformance storage, which enables efficient writing and reading bothfor data that has been stored as deduplicated segments and for data thathas been stored as originally received.

In various embodiments, the storage system comprises a performancestorage system running within the client/user system and the segmentstorage system running in a separate system; multiple performancestorage systems fronting a common segment storage system; a bunch ofapplication servers that have some fast local flash, for very fastaccess, but the segment storage is out on the network and replicated, orany other appropriate configuration of performance and segment systemcomponents running on any appropriate hardware.

Although the following embodiments are described with a capacityoptimized storage comprising a segment storage, any other capacityoptimized storage system can also be used—for example, a storage systemusing compression, using block level deduplication, delta deduplication,or any other appropriate capacity optimization for storage.

FIG. 1A is a block diagram illustrating an embodiment of a storagesystem. In the example shown, storage system 100 receives requests tostore and/or to retrieve one or more data items. In various embodiments,the one or more data items is/are a part of a data stream or data blockor any other appropriate data structure or grouping. Storage system 100receives requests directly or via network 108 from storage user system110. In some embodiments, storage user system 110 is one of a pluralityof systems able to communicate requests to storage system 100.

Storage system 100 comprises system interface 102, performance storagesystem 104, and segment storage system 106. System interface 102receives requests to store data items to and/or retrieve data items fromstorage system 100. System interface 102 passes retrieved data items tonetwork 108 and/or storage user system 110. System interface 102 passesdata items from network 108 and/or storage user system 110 to be storedby storage system 100.

Data items are stored using performance storage system 104. Performancestorage system 104 is able to respond efficiently to requests for datato be stored and to be retrieved. Data items are transferred fromperformance storage system 104 to segment storage system 106. In someembodiments, segment storage system 106 receives data items fromperformance storage system 104, and the data items are processed tobreak data items into segments and to deduplicate the segments (e.g., todetermine whether each segment has been previously stored, and in theevent that the segment has been previously stored, storing a referenceinstead of storing the segment as received). In some embodiments, dataitems after being stored in segment storage system 106 are transferredback to performance storage system 104 and stored as a set of segments.In some embodiments, the original data items, stored as originallyreceived, are removed from performance storage system 104 after the dataitems are transferred back to performance storage system 104.

In some embodiments, segment storage system 106 comprises a storagesystem that compresses segments (e.g., Lempel-Ziv compression,Lempel-Ziv-Welsh compression, Lempel-Ziv-Renau compression, etc.).

Data items that are requested via system interface 102 are retrievedfrom performance storage system 104 or segment storage system 106. Insome embodiments, in the event that there are many requests for a dataitem that is stored in segment storage system 106 and not performancestorage system 104, the data item may be transferred to performancestorage system 104 so that storage system 100 response performance isimproved.

In various embodiments performance storage system 104 comprises flashmemory storage, a fiber channel storage device, persistent memorystorage, a block addressable memory (e.g., 512K block addressablesimilar to a hard drive storage block, in contrast to a byte addressablememory or storage, fixed sized block addressing, and/or logicaladdressable memory), a file system organized memory, random accessmemory, or any other appropriate storage system or system component. Insome embodiments, performance storage system 104 includes data stored asboth a compressed data representation (e.g., a set of deduplicatedsegments) and an uncompressed data representation.

In various embodiments, the Segment Storage System 106 comprises acluster of systems which together provide the functionality of theSegment Storage System.

In various embodiments, segment storage system 106 stores data item(s)using variable length segments, has content addressable segments (e.g.,using a fingerprint of the segment), is replicated using replica segmentstorage system 112 (e.g., stores snapshots and/or segments as stored bysegment storage system 106), is replicated according to a replicationprotocol (e.g., according to a time schedule, after a predeterminedamount of data is stored, or any other appropriate protocol), has a datarepresentation different from performance storage system 104 (e.g., setof segments vs. original data, content addressing vs. logical orphysical addressing, variable block representation vs. fixed block,compressed vs. uncompressed data), or any other appropriatefunctionality for a segment storage system. In various embodiments,segment storage system 106 is replicated by a plurality of replicasegment storage systems similar to replica segment storage system 112that each replicate a portion of or all of the data items and/orsnapshots as stored by segment storage system 106.

FIG. 1B is a block diagram illustrating an embodiment of a storagesystem. In some embodiments, the storage system of FIG. 1B is similar tothe storage system of FIG. 1A with, however, a different partitioning offunctionality in the different system locations—for example, the SystemInterface and Performance Storage System comprise a subsystem withinStorage User System and communicate with the Segment Storage System overa Network. In some embodiments, there may be several such PerformanceStorage Subsystems running in several different Storage User Systems,all communicating with a single Segment Storage System. In the exampleshown, storage system 150 receives requests to store and/or to retrieveone or more data items. In various embodiments, the one or more dataitems is/are a part of a data stream or data block or any otherappropriate data structure or grouping. Storage system 150 receivesrequests directly from storage user system 160. In some embodiments,storage user system 160 is one of a plurality of systems able tocommunicate requests to storage system 150. Storage user system 160includes performance storage system 154 which combined with segmentstorage system 150 comprises a high-speed and high-efficiency datastorage system. In various embodiments, performance storage system 154comprises a high speed solid state memory, a RAM disk, a high speedmagnetic memory, or any other appropriate memory or combination ofmemories.

Storage system 150 comprises system interface 152, performance storagesystem 154, and segment storage system 156. System interface 152receives requests to store data items to and/or retrieve data items fromstorage system 150. System interface 152/performance storage system 154pass retrieved data items to network 158 and to segment storage system156.

Data items are stored using performance storage system 154. Performancestorage system 154 is able to respond efficiently to requests for datato be stored and to be retrieved. Data items are transferred fromperformance storage system 154 to segment storage system 156. In someembodiments, segment storage system 156 receives data items fromperformance storage system 154, and the data items are processed tobreak data items into segments and to deduplicate the segments (e.g., todetermine whether each segment has been previously stored, and in theevent that the segment has been previously stored, storing a referenceinstead of storing the segment as received). In some embodiments, dataitems after being stored in segment storage system 156 are transferredback to performance storage system 154 and stored as a set of segments.In some embodiments, the original data items, stored as originallyreceived, are removed from performance storage system 154 after the dataitems are transferred back to performance storage system 154.

In some embodiments, segment storage system 156 comprises a storagesystem that compresses segments (e.g., Lempel-Ziv compression,Lempel-Ziv-Welsh compression, Lempel-Ziv-Renau compression, etc.).

Data items that are requested and are retrieved from performance storagesystem 154 or if not present in performance storage system 154 fromsegment storage system 156. In some embodiments, in the event that thereare many requests for a data item that is stored in segment storagesystem 156 and not performance storage system 154, the data item may betransferred to performance storage system 154 so that responseperformance is improved.

In various embodiments performance storage system 154 comprises flashmemory storage, a fiber channel storage device, persistent memorystorage, a block addressable memory (e.g., 512K block addressablesimilar to a hard drive storage block, in contrast to a byte addressablememory or storage, fixed sized block addressing, and/or logicaladdressable memory), a file system organized memory, random accessmemory, or any other appropriate storage system or system component. Insome embodiments, performance storage system 154 includes data stored asboth a compressed data representation (e.g., a set of deduplicatedsegments) and an uncompressed data representation.

In various embodiments, segment storage system 156 stores data item(s)using variable length segments, has content addressable segments (e.g.,using a fingerprint of the segment), is replicated using replica segmentstorage system 162 (e.g., stores snapshots and/or segments as stored bysegment storage system 156), is replicated according to a replicationprotocol (e.g., according to a time schedule, after a predeterminedamount of data is stored, or any other appropriate protocol), has a datarepresentation different from performance storage system 154 (e.g., setof segments vs. original data, content addressing vs. logical orphysical addressing, variable block representation vs. fixed block,compressed vs. uncompressed data), or any other appropriatefunctionality for a segment storage system. In various embodiments,segment storage system 156 is replicated by a plurality of replicasegment storage systems similar to replica segment storage system 162that each replicate a portion of or all of the data items and/orsnapshots as stored by segment storage system 156.

FIG. 2 is a block diagram illustrating an embodiment of a segmentstorage system. In some embodiments, the system of FIG. 2 is used toimplement segment storage system 106 of FIG. 1. In the example shown,segment storage system 200 comprises interface 202, segmenter 204,duplicate eliminator 206, file system 208, index 210, and segmentstorage 212. Segment storage system 200 receives data item(s) usinginterface 202. The data item(s) is/are processed by segmenter 204 andfile system 208. Segmenter 204 breaks the data item(s) into segments. Invarious embodiments, the data item(s) are broken into segments byidentifying segment boundaries using a content-based technique (e.g., afunction is calculated at various locations of a data item, when thefunction is equal to a value or when the value is a minimum, a maximum,or other extrema value relative to other function values calculated forthe data item), a non-content-based technique (e.g., based on data itemproperty—for example, byte length, title, creation date), or any otherappropriate technique. In various embodiments, a segment is restrictedto a minimum and/or maximum length, to a minimum or maximum number ofsegments per data item, or any other appropriate limitation. File system208 processes information to indicate the segment(s) associated with adata item. In some embodiments, a list of fingerprints is used toindicate segment(s) associated with a data item. Index 210 is used tolocate stored segments in segment storage 212.

Duplicate eliminator 206 identifies whether a newly received segment hasalready been stored in segment storage 212. In the event that a segmenthas already been stored in segment storage 212, a reference to thepreviously stored segment is stored instead of storing the newlyreceived segment.

Interface 202 receives a request to retrieve a data item. Interface 202communicates with file system 208 to retrieve the data item. File System208 determines the segments associated with the requested data item,communicates with Index 210 to locate segments stored in Segment Store212, reads appropriate segments stored in segment storage 212, andreconstructs the requested data item. The data item is provided viainterface 202 in response to the request.

In some embodiments, file system 208 uses a tree of pointers (e.g.,fingerprints) to associate a file with data segments. In the event thata data item changes, a different segment will be associated with thedata item, and the pointer changes to nodes in the tree will ripple fromthe bottom to the top of the tree associated with a file name (ordirectory structure) efficiently since the appropriate pointers areeasily identified using the tree structure.

FIG. 3 is a block diagram illustrating an embodiment of a performancestorage system. In some embodiments, the performance storage system ofFIG. 3 is used to implement performance storage system 104 of FIG. 1. Inthe example shown, performance storage system 300 includes interface302, file system 304, performance storage unit 306, performance segmentstorage unit 308, and performance storage transfer manager 310.Interface 302 receives data item(s) to be stored. Data item(s) is/arestored using performance storage unit 306. File system 304 storesinformation regarding data item(s) in order to facilitate retrieval ofdata item(s) from performance storage unit 306. In some embodiments,file system 304 is not present and a remote file system or file systemstored with the data is used to identify files within performancestorage unit 306. Performance storage transfer manager 310 manages thetransfer of data item(s) from performance storage unit 306 to a segmentstorage unit. Performance storage transfer manager 310 also manages thetransfer of data item(s) stored as a set of segments to performancesegment storage unit 308. In some embodiments, after a transfer of adata item stored as a set of segment to performance segment storage unit308, the same data item stored in performance storage unit 306 isremoved. In some embodiments, performance storage transfer manager 310,may use a segmenter and duplicate eliminator to transfer a data itemfrom the performance storage unit 306 to the performance segment storageunit 308 without first transferring the data item to a segment storagesystem.

Interface 302 receives requests to retrieve a data item. Data item isretrieved from performance storage unit 306 or performance segmentstorage unit 308 or segment storage system. In some embodiments,performance storage unit 306 is checked prior to checking performancesegment storage unit 308 in the event that a data item is requested tobe retrieved. In some embodiments, performance segment storage unit 308is checked prior to checking segment storage system. In someembodiments, checking is performed by checking using file system 304.

In some embodiments, performance storage transfer manager 310 transfersdata item(s) to a segment storage system by taking a snapshot of dataitem(s) stored in performance storage unit 306. The snapshot enablesrecall of the data item(s) as stored at the time of the snapshot. Insome embodiments, the snapshot is performed according to a snapshotprotocol (e.g., a protocol indicating that a snapshot occurs at a time,when the performance storage unit is full, when performance storage unit306 has only a certain amount of space still available, when a data itemhas not being accessed within a predetermined time period, whenprocessing bandwidth is available to transfer data item(s) to a dataefficient storage system—for example, deduplication data segmentstorage, compressed storage, etc., or any other appropriate protocol),or when it receives a signal indicating a snapshot should be created.

In some embodiments, segment storage has a complete copy of a snapshotso it can replicate a complete snapshot in capacity-optimized form sothat a remote site has a complete copy of the snapshot which is neededfor disaster recovery.

In some embodiments, a data item stored in the performance storage unit306 is not captured using any snapshot (e.g., the file, or data item,was a temporary file, or data item, and not present in performancestorage unit 306 during a snapshot, the file, or data item, version wascreated after a snapshot and then overwritten before a next snapshot,the file, or data item, was an intermediate update that was present onlybetween two snapshots, etc.). In some embodiments, a snapshot isperformed using a log of changes to data items or files in performancestorage unit 306. In some embodiments, a snapshot is created byretaining the root of a tree of pointers.

In some embodiments, a performance system removes the data item, orfile, after (e.g., at once or at a time after depending on the need forthe data item to remain in the performance system) the data item or filehas been stored or transferred to the segment storage system as a resultof a snapshot.

FIG. 4 is a flow diagram illustrating an embodiment of a process for astorage system. In some embodiments, the process of FIG. 4 isimplemented by storage system 100 of FIG. 1. In the example shown, in400 a data stream or data block is stored using a performance storagesystem, where the data stream or data block comprises one or more dataitems. In some embodiments, the performance storage system stores thedata stream or data block as received (e.g., without compression) and isable to provide rapid storage and recall of the data stream or datablock. In various embodiments, the performance storage system comprisesa flash memory, a fiber channel drive, a semiconductor based memory, apersistent memory, or any other appropriate performance storage. In 402,a snapshot is taken of stored data item(s) using a segment storagesystem, where the snapshot enables recall of the data item(s) as storedat the time of the snapshot. In various embodiments, the snapshot occursaccording to a snapshot protocol comprising taking a snapshotperiodically in time (e.g., every hour, every 12 hours, every day, everyweek, every month, etc.), taking a snapshot when the performance memoryhas stored an addition predetermined number of blocks, bytes, or files,taking a snapshot when a usage of the device is low, taking a snapshotof blocks, bytes, or files that have not recently been accessed, or anyother appropriate protocol.

FIG. 5 is a flow diagram illustrating an embodiment of a process for astorage system. In some embodiments, the process of FIG. 5 isimplemented by performance storage system 104 of FIG. 1. In the exampleshown, in 500 a data stream or data block is stored in a performancestorage unit as data item(s). In 502, it is determined whether dataitem(s) are to be automatically stored in a segment storage system as aset of segments. In various embodiments, the data items(s) to beautomatically stored are one of, a portion of, or are all of the dataitems stored by the performance storage unit, or any other appropriatedata items stored by the performance storage unit. In 504, in the eventthat the data item(s) are not to be stored by the segment storage unit,the process waits and control is passed to 502. In various embodiments,the data item(s) are stored automatically according to a protocol wherethe protocol indicates a schedule (e.g., times, dates, periodicity,etc.) or conditions (e.g., percentage memory full, access frequency,storage unit access loading, processor loading, etc.) when data item(s)are automatically stored. In various embodiments, the data item(s) arestored in response to a received signal. In 506, in the event that dataitem(s) are automatically stored in a segment storage system as a set ofsegments, the data item(s) are removed from being stored in theperformance storage unit. In some embodiments, some or all of the dataitem(s) are not removed from the performance storage unit (e.g., in theevent that access activity is high or processor load for storage systemis high). In 508, the set of segments storing the data item(s) is storedin a performance segment storage unit. In various embodiments, theperformance segment storage unit comprises flash memory, a fiber channelstorage device, a persistent memory, or any other appropriateperformance storage device. In some embodiments, the set of segments forthe data item(s) are stored in the performance segment storage unit inthe event that performance access is desired for the data item(s).

FIG. 6 is a flow diagram illustrating an embodiment of a process forproviding requested data from a storage system. In some embodiments, theprocess of FIG. 6 is implemented by storage system 100 of FIG. 1. In theexample shown, in 600 a request to read data is received. In 602, it isdetermined whether requested data is in a performance storage unit. In604, in the event that the requested data is in the performance storageunit, the requested data is provided from the performance storage unit.In 606, in the event that the requested data is not in the performancestorage unit, it is determined whether the requested data is in theperformance segment storage unit. In 608, in the event that therequested data is in the performance segment storage unit, the requesteddata is provided from the performance segment storage unit. In 610, inthe event that the requested data is not in the performance segmentstorage unit, it is determined whether the requested data is in asegment storage system. In 612, in the event that the requested data isin the segment storage system, the requested data is provided from thesegment storage system. In 614, the data is promoted to a performancesegment storage unit, if appropriate (e.g., if it is determined thatperformance access is desired—for example, multiple requests are likelyfor the data), and the process ends. In 616, in the event that therequested data is not in segment storage system, indicate that therequested data is not stored by storage system, and the process ends.

FIG. 7 is a flow diagram illustrating an embodiment of a process fortaking a snapshot of data item(s) in a performance storage unit. In someembodiments, the process of FIG. 7 is implemented by performance storagetransfer manager 310 of FIG. 3. In the example shown, in 700 a newsnapshot is created in the performance storage unit. In variousembodiments, the snapshot in the performance storage unit includes one,a portion of, or all data items stored by performance storage unit basedon a snapshot protocol (e.g., where the protocol sets conditions orcriteria for inclusion of the data item(s) in a snapshot by usage,memory fullness, accesses, new data item(s), or any other appropriateconditions or criteria). In 702, it is determined whether there is aprior snapshot of performance storage unit. In 704, in the event thatthere is no prior snapshot of the performance storage unit, a copy ofthe new snapshot is stored using a segment storage system. In 706, inthe event that there is a prior snapshot of the performance storageunit, then a delta is determined between the new snapshot and the priorsnapshot. In 708, a copy of the prior snapshot is created and the deltais applied to the copy to create a copy of the new snapshot in thesegment storage system. The copy may be a virtual copy comprising newpointers to the same segments, or it may be a new physical copy or itmay be a combination.

In some embodiments, a tree of pointers that includes segments and theassociated files or data items for all the files or data items of thesnapshot can be used to efficiently identify deltas between snapshots.

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

What is claimed is:
 1. A system for storing data, comprising: aperformance storage unit associated with a user system for storing adata stream or a data block in, wherein the data stream or the datablock comprises one or more data items, wherein the performance storageunit comprises a high-speed data storage unit that responds efficientlyto requests for data to be stored and retrieved; a performance storagetransfer manager associated with the performance storage unit associatedwith the user system for determining whether a stored data item of theone or more data items in the performance storage unit is to beautomatically stored in a segment storage system as a set ofdeduplicated segments, wherein the determining is based at least in parton a protocol, wherein the performance storage transfer manager managesa transfer of the one or more data items to be automatically stored inthe segment storage system over a network; the segment storage system incommunication with the network for storing the stored data item of theone or more data items as a set of segments received from theperformance storage unit over the network, wherein the segment storagesystem deduplicates the set of segments to compress the stored dataitem; a performance segment storage unit associated with the user systemand the performance storage unit for storing the set of deduplicatedsegments in the event that the stored data item has been stored usingthe segment storage system, wherein the performance segment storage unitstores the set of deduplicated segments received from the segmentstorage system over the network in the event the stored data item isdetermined to be desired to be available with high performance, whereinthe performance segment storage unit comprises a high-speed data storageunit that efficiently writes and reads the set of deduplicated segments,wherein the performance storage transfer manager also manages a transferof the set of deduplicated segments over the network back to theperformance segment storage unit.
 2. A system as in claim 1, wherein theperformance storage transfer manager removes the stored data item fromperformance storage unit after that the stored data item is stored bythe segment storage system.
 3. A system as in claim 1, wherein thesegment storage system is replicated using a replica segment storagesystem.
 4. A system as in claim 1, wherein the performance storage unitcomprises one or more of the following: a flash memory, a non-transitoryfiber channel storage device, a persistent memory, a persistent memory,a block addressable memory, or a file system organized memory.
 5. Asystem as in claim 1, wherein a snapshot is performed according to asnapshot protocol.
 6. A system as in claim 1, wherein a second storeddata item of the one or more data items is not captured using anysnapshot.
 7. A system as in claim 1, wherein a snapshot is taken using alog of changes to the one or more data items stored by the performancestorage unit.
 8. A system as in claim 1, wherein the segment storagesystem stores a snapshot using variable length segments.
 9. A system asin claim 1, wherein the segment storage system has content addressablesegments.
 10. A system as in claim 9, wherein the content addressablesegments are addressed using a fingerprint of a segment.
 11. A system asin claim 1, wherein the replica segment storage system stores a copy ofa snapshot of the stored data item.
 12. A system as in claim 1, whereinthe replication of the segment storage system occurs according to areplication protocol.
 13. A system as in claim 1, wherein a datarepresentation in the performance storage unit is not the same as thedata representation in the segment storage system.
 14. A system as inclaim 13, wherein the data representation in the performance storageunit comprises an uncompressed data representation.
 15. A system as inclaim 13, where the data representation in the segment storage systemcomprises a compressed data representation.
 16. A system as in claim 1,wherein the performance segment storage unit includes data stored usingboth a compressed data representation and a non-compressed datarepresentation.
 17. A method for storing data, comprising: storing adata stream or a data block in a performance storage unit associatedwith a user system, wherein the data stream or the data block comprisesone or more data items, wherein the performance storage unit comprises ahigh-speed data storage unit that responds efficiently to requests fordata to be stored and retrieved; determining whether a stored data itemof the one or more data items in the performance storage unit is to beautomatically stored in a segment storage system as a set ofdeduplicated segments by a performance storage transfer managerassociated with the performance storage unit associated with the usersystem, wherein the determining is based at least in part on a protocol,wherein the performance storage transfer manager manages a transfer ofthe one or more data items to be automatically stored in the segmentstorage system over a network; storing the stored data item of the oneor more data items as a set of segments received from the performancestorage unit over the network using the segment storage system, whereinthe segment storage system deduplicates the set of segments to compressthe stored data item; and in the event that the stored data item hasbeen stored using the segment storage system, storing the set ofdeduplicated segments in a performance segment storage unit associatedwith the user system and the performance storage unit, wherein theperformance segment storage unit stores the set of deduplicated segmentsreceived from the segment storage system over the network in the eventthe stored data item is determined to be desired to be available withhigh performance, wherein the performance segment storage unit comprisesa high-speed data storage unit that efficiently writes and reads the setof deduplicated segments, wherein the performance storage transfermanager also manages a transfer of the set of deduplicated segments overthe network back to the performance segment storage unit.
 18. A computerprogram product for storing data, the computer program product beingembodied in a non-transitory computer readable storage medium andcomprising computer instructions for: storing a data stream or a datablock in a performance storage unit associated with a user system,wherein the data stream or the data block comprises one or more dataitems, wherein the performance storage unit comprises a high-speed datastorage unit that responds efficiently to requests for data to be storedand retrieved; determining whether a stored data item of the one or moredata items in the performance storage unit is to be sent over a networkto be automatically stored in a segment storage system as a set ofdeduplicated segments by a performance storage transfer managerassociated with the performance storage unit associated with the usersystem, wherein the determining is based at least in part on a protocol,wherein performance storage transfer manager transfers the one or moredata items to be automatically stored in the segment storage system overa network; storing the stored data item of the one or more data items asa set of segments using the segment storage system, wherein the segmentstorage system deduplicates the set of segments to compress the storeddata item; and in the event that the stored data item has been storedusing the segment storage system received from the performance storageunit over the network, storing the set of deduplicated segments in aperformance segment storage unit, wherein the performance segmentstorage unit stores the set of deduplicated segments received from thesegment storage system over the network in the event the stored dataitem is determined to be desired to be available with high performance,wherein the performance segment storage unit comprises a high-speed datastorage unit that efficiently writes and reads the set of deduplicatedsegments, wherein the performance storage transfer manager manages atransfer of the set of deduplicated segments over the network back tothe performance segment storage unit.